posted 01-07-2008 10:08 AM            

Does anyone have any information about base-rates for admissions of problem behaviors pertaining to LEPET investigation targets and pretesting interview issues?

r

------------------

"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-07-2008 12:00 PM            

I'll contact Mark.

I think your drug survey had a base rate around 50 percent, but I couldn't find it quickly in the old posts.

The quoted part below is from an old post regarding the DoDPI LEPET

quote:

---

1I Before (date), did you where shoes?

2SR Regarding your past, do you intend to answer each of my quesitons truthfully?

3C Are you the type of person who would lie to cover for someone?

4R Have you ever committed any serous crime?

5C Are you the kind of person who would betray a member of your family?

6I Before (date) did you ever drive a car?

7R Are you intentionally withholding and information about your involvement with illegal drugs?

8C Are you the type of person who would lie to stay out of trouble?

9R Are you intentionally falsifying or omitting any information on your applicaton forms?

10C Are you the type of person who would blamed someone else for something you did?

11R Have you committed more thefts that what you have told me?

C12 Would you lie to a supervisor for any reason, to save your job?

---

Are these common targets?

Also, does anyone have any anecdotal impressions about base rates for these types of targets? I know we have little data, so a range of impressions is interesting.

r

------------------

"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-07-2008 12:37 PM            

I'm sending you a monte-carlo simulation of a 4 question MGQT technique. I ran it with 10,000 iterations of 4 questions each, using base-rates of .1 for each question.

The monte-carlo simulation will use random numbers to generate truthful and deceptive spots, according to the specified base-rates, and then use more random numbers to produce question by question scores simulating 3-position point totals for each spot, by standardizing a random number to the mean and distribution of parameter estimates for deceptive and truthful spots which were derived from a bootstrap resample of the single issue cases in the OSS-3 training sample, after transforming the OSS-3 model to score the Kircher features in a nonparametric cumulative model according to the bigger-is-better rule. (breathe)

More later.

r

simulation will usin

seed variables for the monte-carlo are derived from the

------------------

"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-07-2008 12:40 PM            

I am currently gathering data about base-rates for admissions on several problem behavior areas in the arena of law enforcement pre-emplyment testing. I have data on almost 70 applicants so far. The areas are as follows:

Employment History
Fired
Disciplined
Withholding Jobs
Undetected Violations

Illegal Drugs
# of Times Used
Types Used
Recency of Use
Sale
Transportation/Smuggling
Cultivation
Manufacturing
Recreational Use of Prescriptions

Detention
# of Times Arrested
# of Times Detained

Undetected Crimes
Theft
Murder
Assault/Battery
Vandalism
Sex
Fraud
Trespassing

I don't have the data calculated yet... How much data should I collect before I publish my findings?

Ben

posted 01-07-2008 08:16 PM            

That should prove very interesting.

With N=70 you may have enough to gain some preliminary figures that might begin to better inform further investigation efforts.

We are really at the onset stages of understanding base-rates and their contribution to LE officer hiring/training success.

Do you collect information about alcohol related problems? Also, how about gambling problems.

It would be interesting to see a literature search regarding the salient content for background investigation of LE candidates.

It would also be interesting to find cohorts of intermediate (3 to 8 years?) and long term LE officers (10+ years?), and perhaps even training failures and integrity failures to better evaluate the contribution of each data point to outcomes.

----------------------

Some estimates.

Observed rates of passing and failure, in mixed issues (independent RQs) exams will be subject to the multiplication rule for independent probability events. This means that we multiply the inverse of the base-rate of each of the RQs. For convenience, we'll assume the RQs to have equivalent base-rates, though this may not be the case in field situations.

Example 1: High base-rate. Assuming a LEPET exam with 4 RQs of BR = 50% = (1-.5)*(1-.5)*(1-.5)*(1-.5) = .0625 chance that any particular subject is being truthful to all RQs.

Example 2: Moderate base-rate. We'll use 15% and 3 RQs. = (1-.15)*(1-.15)*(1-.15) = 61.4% chance that any examinee is truthful to three questions of moderate BR.

Example 3: Low base-rate. Use 5% (.05) and 2 RQs. = (1-.05)*(1-.05) = 90.5% of examinee's can be expected to be truthful to two low BR questions.

Example 4: Very-low base-rate. 1% (.01) with 2 RQs = (1-.01)*(1-.01) = .98 or

The point is that if we are selecting targets that are salient or relevant to the task of predicting risk or future failure, we should be seeking higher BR behaviors that are correlated with the criterion. The trade-off is that we will find comparatively fewer and fewer persons who are being truthful to all questions.

Last week Donna posted some numbers at anti, regarding LEPET exams. I believe she indicated that approximately 70 percent of her subjects had passed. Assuming that LE applicants are not serious criminals (= moderate base-rates) and three to four questions, that would be about the right ball-park for passing rates.

r

------------------

"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-08-2008 08:56 AM            

In answer to your question, I do collect information about alcohol and gambling-related problems. It will not be shown in the study that I am currently working on since neither our application nor pre-screen currently do not cover those issues, but the information is collected at the polygraph phase, and I will have it for possible evaluation and study.

Ben